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Abstract — A large number of the vehicle accidents occurring 
are mainly due to the driver drowsiness and fatigue. The 
accidents occurring due to these reason are more fatal. 
Development of the driver drowsiness detection is a major 
challenge in the Accident avoidance field. The three possible 
ways of developing the drowsiness detection system are based on 
the body parameter analysis, vehicle parameter analysis, and 
face analysis. We concentrate on the face analysis based 
techniques. 


Index Terms — face analysis, template matching, image 
processing 

I. INTRODUCTION 

Most of the eye-detection literature is associated with face 
detection and face recognition. Direct eye detection methods 
search for eyes without prior information about face location, 
and can further be classified into passive and active methods. 
Most eye trackers require manual initialization of the eye 
location before they can accurately track eye features for 
real-time applications. A method for locating the eyes in static 
images was developed by Kanade in 1973 and has been 
improved by other people over the years. Most of these 
researchers have based their methods on Yuille's deformable 
templates to locate and track eye-features .This method looks 
for the valleys and peaks in the image intensity to search for 
the eyes. Once the location of the eyes is determined, its 
position information is used as a priori knowledge to track the 
eyes in succeeding frames. But it requires the eye template to 
be initialized manually at or below the eye otherwise it detects 
the eyebrow instead of the eye. Hallinan has tried to build an 
automated model for deformable templates the best 
candidates for the eye pair, but in order to make his method 
invariant to scaling, the template is initialized at different 
sizes at various places and the best candidates for the eyes are 
selected. Chowetal make use of the Hough transform in 
combination with the deformable templates to extract 
eye-feature points, but this approach is also time consuming 
as the Hough Transform for various scales had to be applied 
prior to detecting the iris, unless the approximate radius of the 
iris is known in advance. Deng and Lai presented the concept 
of regional force sand regional torque to accurately locate and 
resize the template on the iris even when the iris is in an 
extreme position, and for the correct orientation of the eye 
window before it can successfully locate and track the eyelids. 
All these methods track the eyes from frame to frame by read 
justifying the template of both the iris and the eye contour. 
Tianetal have shown that such an approach is prone to error if 
the eyes blink in the image sequence. 
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II. LITERATURE SURVEY 

“A new System for Driver Drowsiness and Distraction 
Detection” Mehrdad Sabet Reza A. Zoorofi et. al. [1] 
proposed a new system has been presented to monitor and 
detect driver drowsiness and distraction. This system uses 
advance technologies based on computer vision and artificial 
intelligence. For face tracking we employed feedback system 
using LBP, to prevent divergence and losing the target. 
Because of feedback system, detection and tracking modules 
can cooperate with each other. Eye state analysis performed 
by SVM with features which extracted using LBP operator. 
The proposed algorithm for face tracking and eye state 
analysis is shown to be robust and accurate for varying light, 
background changes and facial orientation. The system is also 
observed to provide agreeable results. 

“Sleepy Eye’s Recognition for Drowsiness Detection” 
Shinfeng D. Lin et. al.[2] presents a sleepy eye's recognition 
for drowsiness detection without the training stage. In the 
beginning, an Adaboost classifier with Haar - like features is 
utilized to find out the face area. Then the eyes region is 
located by ASM. Finally, the binary pattern and edge 
detection are adopted to recognize the eye’s state. 
Experimental results prove that the proposed method could 
accurately detect the sleepy eyes. In addition, the comparative 
performance shows that the proposed recognition system 
without the training stage is useful for driver’s drowsiness 
detection. 

“Design and Implementation of Driver’s Drowsiness 
Detection System on Digitalized Driver System-2013” 

Pranto Hidaya Rusmin et. al. [3] proposes a method which 
utilizes both the driver eye drowsiness detection as well as 
vehicle parameter analyses. This method of the driver 
drowsiness detection uses the fact that the eye region is in the 
l/3 rd top region of the face and hence the data overhead to 
detect the eye region in the face is reduced to a great extent. 
Then for drowsiness detection this uses basic template 
matching technique as earlier. 

“Drowsy Driver Warning System using Image 
Processing” Singh himani parmer et. al.[4] proposes a 
drowsiness detection system based on the image processing 
technique. Here the symmetry of the face is utilized an half the 
face is only is used as input data hence reducing the data to be 
processed. After that the region in the face with the maximum 
darkness is identified and this region is the eye region. The 
previous mentioned process is only applied for the first frame 
and for the subsequent frames the eye tracker takes care of 
updating the information about the eye position. If only there 
is large deviation in the eye region the initialization phase is 
repeated. A two circle template matching technique is used 
for detection of state of the eye. 

“Video-Based Detection and Analysis of Driver 
Distraction” Yiaxiao Yun et. al. [5] here in this paper the 
authors talk about both inner as well as outer side of the car to 
be traced. Video recording sensors are used and based on the 
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frames received the status of the driver is drawn. The main 
distinguishing feature of this method is that the authors also 
focus on the outside of the car to track the vehicles on both 
side of the car by using sensors and based on the distance the 
driver is alerted about the situation. Face analysis and 
detection here works in a simple manner by reading the 
frames recorded by the camera and recognising the face. After 
face is recognised the other parts of the face are recognised 
based on the facial geometry, i.e. relative position of various 
organs on the face. 

“Feature extraction” Xiaoguang Lu et.al. [6] 

The process to determine the query face a one to many 
matching process is used which distinguish a query face 
image to all the stenciled images in a given face database. The 
identification process for the test images is carried out by 
detecting the image in the database which has the exorbitant 
resemblance with the test image. The identification process is 
a ’’closed” test, which explains that the sensor gets hold of an 
examination made of an individual that is known to be in the 
database. The features of the test subject is distinguished to 
the database's features and a sense of likeness for each feature 
is established. These similarity scores are then numerically 
ranked in a descending order. A "top match score" is a 
percentage of the number of times the greatest similarity score 
has been the right match for all the individuals. If any of the 
top r similarity scores is compatible to the test subject, it is 
taken into account as a correct match in terms of the 
cumulative match. The number of times one of the r similarity 
scores is the correct match for all individuals, is referred to as 
the ’’Cumulative Match Score”. The ’’Cumulative Match 
Score” curve is the rank "n" to the percentage of the right 
recognition, where rank n is defined as the number of the 
highest similarity score. 

“Kernel Principle Component Analysis in Face 
Recognition System: A Survey” Ritu Upadhayay et.al. 
[7] This step for Face Detection is to examine, if the human 
faces appear in a given image, and to locate it. The 
approximate results of this process are patches containing 
each face in the image input. With the purpose of [2] making 
further face recognition system more simple to carry it out, 
face configuration are executed to explain the scales and 
alignments of these patches. 

The Feature Extraction step comes after the face detection 
step, the human-face patches are extracted from images. 
Using these patches instantly for face recognition have a few 
downsides to it, firstly each patch comprises over one 
thousand pixels but are too large to construct [2] a sturdy 
recognition system. Secondly, the face patches may be taken 
up from a different camera orientations and with a different 
face expressions as well as illuminations and may be afflicted 
from occlusion and clutter. To overcome with these 
drawbacks, the feature extractions are carried out to do 
in-formation packing, dimension reduction, salience 
extraction, and noise cleaning. 

The face recognition is used for two key functions: [3] [4], 
Firstly, it is verification (one-to-one matching) when it is 
presented with an unknown individual’s face image alongside 
with a claim of identity, determining whether the individual is 
who he or she declares to be. Secondly, it is the identification 
(one-to-many matching), where an image of of an unidentified 
individual is provided with and examining that person’s 
identity by distinguishing that image with a database of the 
images of known individuals. There are a number of 


application areas in which face recognition can benefit from 
confirmation and recognition. 

“Multi-view face detection based on kernel principal 
component analysis and kernel support vector 
technique.” Muzhir Shaban Al-Ani et. al. [8] The algorithm 
can be obtained upon the capability of the KPCA. The KPCA 
feature removal efficaciously acts a nonlinear mapping from 
the input space to an implicit high dimensional feature space. 
The steps needed to carry out the principal components can be 
briefed up as follows; 

• the matrix K can be computed, see Eq. (2). In this paper the 
polynomial kernel is used as a kernel function: ",$% 
"&".$%'(%). 

Where a = 0.001; b =-l; n = 3 and xk RN are taken from the 
face images by rearranging the pixel value order as shown • In 
order to obtain the eigenfaces, the face image data are 
converted from matrices to vectors, where the vector version 
of each face is a column in a matrix. • The resulting 
eigenfaces are then point multiplied with the training set 
images to filter out outlier data and focus the training on the 
principal features of the face. The resulting images have their 
intensities scaled. • The first M = 50 most notable principal 
components are used as the basic vectors. It aims at training 
the KSVC to distinguish between the face and non-face 
patterns for face detection. • Projections of a test point onto 
the eigenvectors are computed. 

“Automatic Facial Expression Analysis A Survey” 

C.P. Sumathi et.al. [9] 

This system evaluates the measurement of actions and also 
divides the actions as per the manual of FACS [9]. The 
procedural steps involved in the analysis are l.Face 
Acquisition 2.Facial Expression Extraction 3.Expression 
Recognition. 

Face acquisition is the first step in which faces are detected 
from the input images or image sequences. The Face 
acquisition may detect faces in input images or detect face in 
the first frame and track the face in the remaining frames in 
case of image sequences. Based on the type of input the faces 
can be classified into 2D or 3D faces. After the face is located 
then thefacial features are extracted to identify the facial 
expression. Facial expression can be classified into two types 
namely Geometric or In Transient features and Appearance 
Features or Transient Features. 

Geometric or Intransient Features: The features that are 
always present in the face but may be deformed due to any 
kind of facial expression.eg)Eyes, Eyebrows, Mouth, Tissue 
Textures, Nose. The facial components or facial feature 
points are extracted to form a feature vector that represents 
the face geometry 

Appearance or transient Features: The features that appear 
temporarily in the face during any kind of Facial Expression. 
Eg) Different kinds of wrinkles, bulges, forefront, regions 
surrounding the mouth and eyes. With the help of the 
appearance-based methods, image filters, such as Gabor 
wavelets, are applied to either the whole-face or specific 
regions in a face image to extract a feature vector. 

The last step is the Facial Expression Recognition in facial 
expression analysis where the extracted features are 
determined based on the action units. Not only the basic 
emotions like anger, happiness, sorrow are identified by the 
Recogniser but also identifies the expression caused due to 
pain, temporal dynamics , Intensity of Expression, 
Spontaneous Expression. 
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“Multi-View Algorithm for Face, Eyes and Eye State 
Detection-study paper” Latesh Kumari et.al [10] 

This involves the Skin colour segmentation, wherein with the 
help of the skin colour segmentation we can separate the skin 
and non-skin regions of the image. This is however dependent 
of the colour models in which we are illustrating the human 
image. The selection of the colour model plays a significant 
role in skin segmentation to make it more effectual. The 
prospective colour spaces: □ CIEXYZ □ CIEXYZ □ YCbCr 
□ YUV □ YIQ .The computation of the separability of 
clusters of skin and non-skin pixels using scatter matrices. 
Another metric is to do a histogram comparison of the skin 
and non-skin pixels after colour space transformation. The 
YCbCr colour space was found to perform very well. 

Skin detection is defined as a process of removal of skin 
pixels and non-skin pixels in an image or a video. The skin 
and non-skin regions are obtained by dividing the pixels of the 
given image with the help of a detector. According to 
computer vision, there are many ways to approach the 
methods developed for skin detection. A skin detector 
typically transforms a given pixel into an appropriate color 
space and then uses a skin classifier to label the pixel whether 
it is a skin or a non-skin pixel. In any given color space, skin 
color occupies a part of such a space, which might be a 
compactor large region in the space. Such region is usually 
called the skin color cluster. A skin classifier is a one-class or 
two-class classification problem. A given pixel is classified 
and labelled whether it is a skin or a non-skin given a model of 
the skin color cluster in a given color space. In the context of 
skin classification, true positives are skin pixels that the 
classifier correctly labels as skin. True negatives are non-skin 
pixels that the classifier correctly labels as non-skin. Any 
classifier makes errors: it can wrongly label a non-skin pixel 
as skin or a skin pixel as a no skin is a performance. The 
former type of errors is referred to as false positives (false 
detections) while the later is false negatives. A good classifier 
should have low false positive and false negative rates. As in 
any classification problem, there is a trade-off between false 
positives and false negatives. The more loose the class 
boundary, the less the false negatives and the more the false 
positives. 

Eye detection: Eyes are detected based on the assumption that 
they are darker than other part of the face. Han at al. use 
morphological operations to locate eye-analogue segments, 
while Wu and Zhou find eye-analogue segments searching 
small patches in the input image that are roughly as large as an 
eye and are darker than their neighbourhood’s. For eye 
detection, there are four methods are used which include 
Viola-Jones eyes detector, the algorithm described by Valenti 
and Gevers , the approach proposed by Timm and Barth, and 
feature extractor by Ribaric, Lovrencic, and Pavesic. 
“Automatic Eye Detection and Its Validation” Peng Wang 
et.al. [11] 

This paper talks about the Eye Detection Error on Face 
Recognition and the observation of how the recognition 
performance varies according to eye localization error and the 
eye positions of the base truth are synthetically concerned 
with random noises. The Face recognition is carried out using 
the perturbed eye positions. Given a normalized error, the 
random noise is uniformly distributed at a circle in 2D space. 
The data provided from FRGC 1.0 and PCA baseline 
algorithm are used in the experiment .The eye location errors 
notably influence the recognition accuracy. For example, 


about 1% (about 3 pixels for FRGC image or 0.5 pixel if the 
interocular distance is 50 pixels) eye location error reduces 
the face recognition accuracy by over 10%. When the error is 
about 5%, the face recognition accuracy reduces by 50%. 
Drowsy Driver Warning System Using Image 
Processing” Singh Himani Parmar et.al. [12] 

The Localization of Face takes place based on the symmetry 
of the face, we use a symmetry-based approach. A sub 
sampled, gray-scale version of the image was found to be 
enough for the usage. A symmetry-value is then computed for 
every pixel-column in the reduced image. Consider the image 
to be denoted by I(x, y) then the symmetry value for a 
pixel-column is given by S(x) = XX [abs I ((x,y-w)-(x,y+w))]. 
S(x) is computed for X € [k,size-k] where k is the maximum 
distance from the pixel-column that symmetry is measured, 
and x size is the width of the image. The x corresponding to 
the lowest value of S(x) is the center of the face. 

The Location of Eyes consists of a raster scan algorithm 
which is used for the most exact location of the eyes and 
extracts that vertical location of eyes. 

The eye is tracked by looking for the darkest pixel in the 
predicted region. To help to be able to overcome tracking 
errors, we make sure that not even a single geometrical 
constraint is violated. Incase they are violated , we relocalize 
the eyes in the next frame. To find the best match for the eye 
template, we initially center it at the darkest pixel, and then a 
gradient descent is performed in order to find a local 
minimum. 

Detection of Drowsiness talks about when the driver becomes 
more fatigued, the driver would have more eye-blinks to last 
longer. The number of consecutive frames that the eyes are 
closed will be kept in count in order to decide the condition of 
the driver. We need to determine if the eyes would be opened 
or closed; so a method is used that looks at the horizontal 
histogram across the pupil. 

“Driver Fatigue Detection based on Eye State Analysis” 
Yong Du PeijunMa Xiaohong Su Yingjun Zhang et.al.[13] 
Localization of the Face consists of when the driver’s eyes’ 
position and fight conditions that changes constantly while 
driving on road. It is not easy to search And locate the driver’s 
eye at a constant location. Moreover, the background is 
usually complex and unpredictable especially when the 
automobile is running on the road. The first step is to detect 
the face which helps to further reduce the range of eye 
detection and the process will improvise the accuracy and the 
speed of the eyes location and lower the interference of 
background. For purpose of reducing the blindness of face 
searching, we calculate interframe differences to decide 
whether there is an moving object. When a given object is 
detected and in the local area there contains skin color 
information in YCbCr color space, it is believed that there is a 
person before the camera. Although people of different races 
are different in skin color, a planar Gaussian distribution can 
be used to approximate the distribution of human skin color in 
YCbCr color space. 

The localization of the eyes talks about the thermodynamic 
principle, for a closed system, the minimum of Helmholtz 1 
free energy f E is used to describe the equilibrium state in 
stead of maximum entropy. In such a system, the particles of 
set i has the same free energy of i f E. By Taking into 
consideration the vast variety to the nearby pixels and the 
likeness to the eye pixels sufficiently, the specific region of 
the eyes can be obtained 
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"Driver Drowsiness Detection System and Techniques: A 
Review Research” Vandna Saini et.al. [14] 

Optical detection is the most common application of an 
optical sensor system which uses infrared or near-infrared 
LEDs to light the driver’s pupils and are then observed by a 
camera system. The computer algorithms examine the blink 
rate and duration to determine drowsiness. The camera 
system may also monitor The facial features and head 
positions are also observed by the camera system for any signs 
of drowsiness, such as yawning and sudden head nods. 

Most of the researchers have taken into account the following 
physiological signals to detect drowsiness, Electrocardiogram 
(ECG), electroencephalogram (EEG). The different stages of 
drowsiness, can be varied significantly due to the heart rate. 
Therefore, heart rate, which can be easily determined by the 
ECG signal, can also be used to detect drowsiness. Others 
have measured drowsiness using Heart Rate Variability 
(HRV), in which the low (LF) and high (HF) frequencies fall 
in the range of 0.04-0.15 Hz and 0.14-0.4. 

“Performance Analyses and Comparison of Eye Detection 
Techniques” Vijayalaxmi et.al. [15] 

Using the Neural Network method the driving support 
systems, are becoming regular and aid the driver in many 
ways. A non-intrusive method of detecting Fatigue and 
drowsiness based on eye-blink count and eye directed 
instruction control helps the driver to avoid collisions caused 
due to driver’s drowsiness. Eye detection and tracking 
methods under different circumstances such as illumination, 
background, face alignment and facial expression makes the 
concepts difficult. The algorithm proposed describes that 
firstly the network is trained to reject the non-eye region 
based on images with features of eyes and the images with 
features of non-eye using Gabor filter and Support Vector 
Machines to reduce the dimension and classify efficiently. 
The face is firtly divided using L*a*b transform colour space 
and then is later detected using HSV and Neural Network 
approach. The first step in the proposed algorithm is training 
SVM . A large data of eye and non-eye patterns are collected 
to train SVM. The images containing eyes of various sizes, 
alignments, positions, whether it is opened or closed, the 
non-eye-pair images are cut appropriately for the training of 
SVM. The right way of choosing a non-eye-pair image is 
highly crucial to train SVM because performance of SVM is 
dependent on the kind of non-eye-pair images that are being 
used. Initially the training SVM, the non-eye-pair images are 
similar to eye pair such as eyebrows, nostrils and other 
eye-pair like patches as eye pair. The performance of the 
method is determined with the sensitivity rate of 88.6%, 
specificity of 95.2% and accuracy of 89.2%. 

“A Computer Vision-Based System for Real-Time 
Detection of Sleep Onset in Fatigued Drivers” Alexandra 
Branzan Albu et.al. [16] 

In a given frame, template matching is not performed with a n 
exhaustive search. The first location of the match wherein the 
likeness of the ratio x exceeds the tolerance rR is retained 
which helps in the speed of computation. 

The closed-eye states are not as frequent as the open-eye 
states are, so template matching for open-eye is carried out 
initially. If no match is found for open eyes, a matching 
process against closed-eyes is carried out later on. If both 
matching processes do not identify matches then decision of 
no match is declared thereby. 


“Fighting Accident Using Eye Detection for Smartphones 
Monitoring Physiological Characteristics” Mohamed A. 
Mohamed et.al.[17] 

Inorder to obtain the best possible accurate method we can 
use the ones based on human physiological phenomena. It can 
becarried out in two ways; firstly to measurethe changes in 
physiological signals, suchas brain waves or heart rate or to 
measure physical changes such as eye blinking or sagging 
posture, leaning of the driver’s head and the open or closed 
states of the eyes.To measure the changes in physiological 
signals is not realistic, since the sensing of electrodes would 
have to be attached directly onto the driver’s body, and would 
inturn be a distraction for the driver. 

If thedriver would have to drive for a long time it would result 
in perspiration on the sensors; lowering their ability to 
monitor accurately. The IR sensors & camera module that is 
to be used for measuring the changes in physiological signals 
brain waves. The second technique which has been discussed 
and executed in this paper is well suited for real world driving 
conditions since it can be non-intrusive by using optical 
smartphone cameras to detect change. 

“Sleepy Eye's Recognition for Drowsiness Detection” Lin 
.S.D et.al. [18] 

The heavy traffic often is a major cause that leads to a huge 
number of accidents taking place. The most common 
accidents taking place these days is due to the driver’s 
drowsiness. To avoid this situation, this paper proposes a 
sleepy eye's recognition system for drowsiness detection. 
Initially, a cascaded Adaboost classifier with the Haar-like 
features is made use of to find out the face region. Secondly, 
the eye region is located by Active Shape Models(ASM) 
search algorithm. The binary pattern is then used and edge 
detection are adopted toobtain the feature of the eyes and 
determinethe eye's state. Experimental results demonstrate the 
comparative performance, even without the training stage, 
with other methods. 

“A Survey on Driver’s Drowsiness Detection Techniques” 
Jay D. Fuletra et.al. [19] 

The main cause for fatal accidents to take place is the 
Drowsiness and sleeping while driving. The different 
drowsiness detection techniques are divided and then 
distinguished using their features. The computer vision based 
image processing techniques is one of them. It uses different 
kinds of images of the driver to detect drowsiness states using 
his/her eyes states and facial expressions. This technique is on 
the focus of this survey paper. 

“Driver Fatigue Detection based on Eye State Analysis” 
Yong Du Peijun Ma et.al. [20] 

This paper presents an effective aspect based on the driver 
fatigue detection method. Firstly, the inter-frame difference 
can be used to detect the face that binds the colour 
information. If it exists so then the face area is divided from 
the basis of a mixture of a skin tone model. Then the process 
of crystallization to obtain the location of eyes within face 
area can be carried out, ahead of this, the different features of 
the eye can be used to examine the eye’s status. Finally, the 
driver fatigue is validated by examining the changes of eye’s 
states. The proposed method, results show validity of the 
experiments carried out. 
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III. CONCLUSION 

The different papers for sleepy eye detection and drowsiness 
detection papers are discussed here. The main focus here is on 
the face analysis based drowsiness detection system. To 
conclude even after so much study we feel that sleepy eye 
detection is an open area for research. 
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